PATENT ABSTRACTS OF JAPAN 
(1 1 publication number : 2002-029349 
(43)Date of publication of application : 29.01 .2002 



(51)Int.CI. B60R 21/00 

B60R 1/00 

G08G 1/16 

H04N 5/225 

H04N 7/18 



(21 Application number : 2000-21 221 6 (71 Applicant : NISSAN MOTOR CO LTD 
(22)Date of filing : 1 3.07.2000 (72)Inventor : ANDO TOSHIYUKI 



(54) DEVICE FOR RECOGNIZING VEHICULAR CIRCUMFERENCE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To accurately and inexpensively recognize an obstacle 
having the possibility of contact with a vehicle, and a distance upto the obstacle. 
SOLUTION: A position for measuring a distance is set within a photographing range of 
an image pick-up means for imaging a vehicular circumference, and a focal point of a 
photographing lens of the image pick-up means is adjusted in a range finding position 
based on an image signal from the image pick-up means. A distance upto the range 
finding position is calculated based on a position of the photographing lens and a focal 
distance thereof when focus- adjusted in the range finding position, calculated 
distance information is composed to an image of the vehicular circumference 
image-picked up by the image pick-up means to be displayed. The distance upto the 



obstacle is accurately recognized without using an expensive distance measuring 
instrument. 



* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect 
the original precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A vehicle circumference recognition device comprising: 
An imaging means which picturizes a vehicle circumference. 

A ranging positioning means to set up a position which measures distance in an 
imaging range of said imaging means. 

A focus means to which said ranging position is made to carry out the focus of the 
taking lens of said imaging means based on a video signal from said imaging means. 
A displaying means which compounds and displays distance information computed by 
said distance calculation means on an image of a vehicle circumference picturized by 
distance calculation means which computes distance to said ranging position based on 
a position and a focal distance of said taking lens when a focus is carried out to said 
ranging position, and said imaging means. 

[Claim 2]A vehicle circumference recognition device installing said imaging means in a 
part which vehicles projected most in the vehicle circumference recognition device 
according to claim 1 so that an optic axis of said taking lens may become a road 
surface and parallel. 

[Claim 3]A vehicle circumference recognition device installing said imaging means in a 
bumper of a vehicle rear in the periphery recognition device for vehicles according to 
claim 2. 

[Claim 4]A vehicle circumference recognition device, wherein said ranging positioning 
means sets up a ranging position for every prescribed interval on Chuo Line of a 
sliding direction of an imaging range of said imaging means in the vehicle 



circumference recognition device according to claim 2 or 3. 

[Claim 5]A vehicle circumference recognition device, wherein said displaying means 
connects and displays each ranging position in the vehicle circumference recognition 
device according to claim 4 by a line segment of a color according to distance to each 
ranging position. 

[Claim 6]A vehicle circumference recognition device, wherein said displaying means 
connects and displays each ranging position in the vehicle circumference recognition 
device according to claim 4 by a line segment of thickness according to distance to 
each ranging position. 

[Claim 7]In the vehicle circumference recognition device according to claim 4, based 
on distance to each ranging position computed by said distance calculation means, A 
vehicle circumference recognition device characterized by what it has a 
closest-approach point calculating means which computes a position and distance of 
an obstacle nearest to a periphery of vehicles, and said displaying means compounds a 
position of the nearest obstacle, and information on distance on a periphery of 
vehicles, and displays on an image of a vehicle circumference picturized by said 
imaging means. 

[Claim 8]The vehicle circumference recognition device according to claim 7 which is 
provided with the following and characterized by said closest-approach point 
calculating means computing a position and distance of an obstacle nearest to a 
vehicles periphery of a predicted position. 
A rudder angle detection means to detect a steering angle. 

A position prediction means which predicts a position of a vehicles periphery after 
carrying out prescribed distance movement with the present steering angle. 

[Claim 9]A vehicle circumference recognition device which a taking lens of said 
imaging means is a zoom lens in a vehicle circumference recognition device given in 
one paragraph of claims 1-8, and is characterized by said focus means' raising image 
pick-up magnification of said zoom lens, and performing a focus. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the vehicle circumference recognition 
device which displays the situation of the vehicle circumference used as a driver's 
dead angle on the display of the car interior of a room. 
[0002] 



[Description of the Prior Art] A camera is installed in a vehicle rear, vehicles back is 
picturized, and the vehicle circumference recognition device which displays the image 
of vehicles back on the monitor of the car interior of a room is known (for example, 
refer to JP,10-181447,A). As shown in drawing 1 , vehicles back is picturized and the 
indices 3a and 3b which show the distance from the body 1 are expressed on the 
image 2 of the picturized vehicles back as this device in piles so that a part of body 
(bumper) 1 may be copied with the camera attached to the vehicle rear. The driver 
can read in the range indexes 3a and 3b the distance to the obstacle 4 projected on 
the image 2 of vehicles back. 
[0003] 

[Problem(s) to be Solved by the Invention]However, in the conventional vehicle 
circumference recognition device mentioned above. Since the range indexes 3a and 
3b are displayed on the basis of the road surface, not the actual distance L1 (= 1 m) 
until the bumper 1 contacts the obstacle 4 but the distance L2 (= 2 m) of the index 3b 
which has lapped with the obstacle 4 may be read, and it may take for there being the 
obstacle 4 in the distance rather than the actual condition. When many obstacles exist, 
Since a driver has to read whether there is any possibility that the distance to which 
obstacle may contact which position of an obstacle at the shortest on the basis of the 
road surface displayed on the screen, there is a problem of being hard to recognize 
the physical relationship of vehicles and an obstacle correctly, and taking it more 
often from a screen. 

[0004]In order to solve such a problem, distance measuring equipments, such as a 
laser radar, can be used, and the distance to an obstacle can be detected correctly, 
but since a device becomes expensive, adoption is difficult. 

[0005]The purpose of this invention is to provide the cheap vehicle circumference 
recognition device which can recognize correctly the distance to an obstacle with a 
possibility that vehicles may contact, and its obstacle. 
[0006] 

[Means for Solving the Problem]When it matches with drawing 2 and drawing 26 in 
which composition of 1 embodiment is shown and this invention is explained, an 
invention of (1) claim 1, The imaging means 11 which picturizes a vehicle 
circumference, and a ranging positioning means 15 to set up a position which 
measures distance in an imaging range of the imaging means 11, The focus means 11 
to which a ranging position is made to carry out the focus of the taking lens 11a of the 
imaging means 11 based on a video signal from the imaging means 11, The distance 
calculation means 1 1 which computes distance to a ranging position based on a 
position and a focal distance of the taking lens 11a when a focus is carried out to a 
ranging position, It has the displaying means 14 and 15 which compound and display 
distance information computed by the distance calculation means 1 1 on an image of a 
vehicle circumference picturized by the imaging means 11, and this attains the 



above-mentioned purpose. 

(2) Install a vehicle circumference recognition device of claim 2 in a part to which 
vehicles projected the imaging means 1 1 most so that an optic axis of the taking lens 
1 1a may become a road surface and parallel. 

(3) A periphery recognition device for vehicles of claim 3 installs the imaging means 1 1 
in a bumper of a vehicle rear. 

(4) A vehicle circumference recognition device of claim 4 sets up a ranging position for 
every prescribed interval on Chuo Line of a sliding direction of an imaging range of the 
imaging means 11 by the ranging positioning means 15. 

(5) By the displaying means 14 and 15, a vehicle circumference recognition device of 
claim 5 connects each ranging position with a line segment of a color according to 
distance to each ranging position, and display it. 

(6) By the displaying means 14 and 15, a vehicle circumference recognition device of 
claim 6 connects each ranging position with a line segment of thickness according to 
distance to each ranging position, and display it. 

(7) Based on distance to each ranging position which a vehicle circumference 
recognition device of claim 7 computed by the distance calculation means 11, It has 
the closest-approach point calculating means 15 which computes a position and 
distance of an obstacle nearest to a periphery of vehicles, and a position of an 
obstacle nearest to a periphery of vehicles and information on distance are 
compounded on an image of a vehicle circumference picturized by the imaging means 
1 1 , and it is made to display on it by the displaying means 1 4 and 1 5. 

(8) A vehicle circumference recognition device of claim 8 is provided with a rudder 
angle detection means 50 to detect a steering angle, and the position prediction 
means 15 which predicts a position of a vehicles periphery after carrying out 
prescribed distance movement with the present steering angle, and by the 
closest-approach point calculating means 15. A position and distance of an obstacle 
nearest to a vehicles periphery of a predicted position are computed. 

(9) The taking lens 11a of the imaging means 1 1 of a vehicle circumference recognition 
device of claim 9 is a zoom lens, and as for the focus means 1 1 , image pick-up 
magnification of a zoom lens is raised, and be made to perform a focus. 

[0007]In a paragraph of The means for solving a technical problem mentioned above, 
in order to explain plainly, a figure of 1 embodiment was used, but thereby, this 
invention is not limited to 1 embodiment. 
[0008] 

[Effect of the Invention](1) According to the invention of claim 1, set up the position 
which measures distance in the imaging range of the imaging means which picturizes a 
vehicle circumference, and carry out the focus of the taking lens of an imaging means 
to a ranging position based on the video signal from an imaging means. And since the 
distance information which computed the distance to a ranging position based on the 



position and focal distance of a taking lens when a focus is carried out to a ranging 
position, and was computed on the image of the vehicle circumference picturized by 
the imaging means is compounded and it was made to display, The distance to an 
obstacle can be recognized correctly, without using expensive distance measuring 
equipment. 

(2) According to the invention of claim 2, since the imaging means was installed in the 
part which vehicles projected most so that the optic axis of a taking lens might 
become a road surface and parallel, the distance to an obstacle with a possibility that 
the lobe of vehicles may contact can be recognized correctly. 

(3) Since the imaging means was installed in the bumper of a vehicle rear according to 
the invention of claim 3, the distance to the obstacle of vehicles back with a 
possibility that the rear bumper of vehicles may contact can be recognized correctly. 

(4) Since the ranging position was set up for every prescribed interval on Chuo Line of 
the sliding direction of the imaging range of an imaging means according to the 
invention of claim 4, the distance to all the obstacles with a possibility that the part 
which vehicles, such as a bumper, projected most may contact can be recognized 
correctly. 

(5) According to the invention of claim 5, since each ranging position is connected 
with the line segment of the color according to the distance to each ranging position 
and it was made to display, it becomes easy to recognize the obstacle near vehicles 
visually. 

(6) According to the invention of claim 6, since each ranging position is connected 
with the line segment of the thickness according to the distance to each ranging 
position and it was made to display, it becomes easy to recognize the obstacle near 
vehicles visually. 

(7) According to the invention of claim 7, based on the distance to each ranging 
position, the position and distance of the obstacle nearest to the periphery of vehicles 
are computed, Since the position of the obstacle nearest to the periphery of vehicles 
and the information on distance are compounded on the image of the vehicle 
circumference picturized by the imaging means and it was made to display on it, The 
position of the high obstacle of a possibility that a vehicles periphery will contact, and 
the distance to the obstacle can be recognized at a glance, it is easy to hold the 
distance to an obstacle, and contact with an obstacle can be avoided certainly. 

(8) Since according to the invention of claim 8 the position of the vehicles periphery 
after carrying out prescribed distance movement with the present steering angle is 
predicted and the position and distance of the obstacle nearest to the vehicles 
periphery of a predicted position were computed, After carrying out prescribed 
distance movement with the present steering angle, the position of the high obstacle 
of a possibility that a vehicles periphery will contact, and the distance to the obstacle 
can be recognized at a glance, and contact with an obstacle can be avoided certainly. 



(9) Since according to the invention of claim 9 the taking lens of an imaging means is 
used as a zoom lens, the image pick-up magnification of a zoom lens is raised and it 
was made to perform a focus, The boundary of the color of an image can become clear, 
distribution of the high frequency component contained in a video signal can become 
steeper, and a focus can be carried out to a ranging position more correctly than the 
camera of the taking lens of fixed focus distance, therefore the distance to a ranging 
position can be found more correctly. 
[0009] 

[Embodiment of the Invention]«A 1 st embodiment of an invention» Drawing 2 is a 
figure showing the composition of a 1 st embodiment. The back camera 1 1 is provided 
with the taking lens 11a (refer to drawing 7 ) and a CCD sensor (un-illustrating). is 
attached to a vehicle rear, and picturizes vehicles back. This camera 1 1 is provided 
with the automatic focusing function, can carry out the focus of the taking lens 1 1 a to 
the arbitrary positions of an imaging range, i.e., a field, can double a focus, and detects 
the object distance to a focus position based on the focal distance and lens position 
of the taking lens 11a. According to this embodiment, the object distance to the focus 
position detected with the camera 1 1 is used as a distance to the obstacle of vehicles 
back. How the automatic focusing function of a camera detects object distance is 
mentioned later. 

[0010]The shift sensor 12 detects the setting-out position of the shift lever of a 
gearbox (un-illustrating). One [ the ignition switch 13 ] if the ignition key of vehicles is 
set as on position. The monitor 14 displays the distance information to the obstacle of 
vehicles back while displaying the image of the vehicles back picturized with the 
camera 1 1 . 

[0011]The controller 15 is provided with periphery articles, such as CPU15a and the 
memory 15b, and performs image pick-up control of the camera 11, and display 
control of the monitor 14. The position in which the controller 15 tends to measure 
distance in the imaging range of the camera 1 1 . It sets up (it is hereafter called a 
ranging position), the object distance to the ranging position detected with the camera 
1 1 is inputted, and it compounds and displays on the vehicles back image of the 
monitor 14 as distance information to the obstacle of vehicles back. 
r0012l Drawing 3 shows the imaging range of the camera 11 seen from the vehicles 
side in a motor pool, and drawing 4 shows the imaging range of the camera 1 1 seen 
from the vehicles upper part in a motor pool. According to this embodiment, the 
camera 1 1 is installed in the bumper which the vehicle rear projected most so that the 
optic axis 21 of the taking lens 11a may become a road surface and parallel. The 
vertical imaging range 22 (refer to drawing 3 ) and the horizontal imaging range 23 
(refer to drawing 4 ) of the camera 1 1 are decided with the taking lens of the camera 
1 1 , etc. 24 is a wall of a motor pool. 

r0013l Drawing 5 is an image of the vehicles back picturized with the camera 1 1 in the 



motor pool shown in drawing 3 and drawing 4 . Since the camera 1 1 is installed so that 
the optic axis 21 may become a road surface and parallel against the bumper of a 
vehicle rear, the image of the vehicles back picturized with the camera 1 1 turns into 
an image seen from the height of the bumper. 

[0014] Drawing 6 is a flow chart which shows the image processing program executed 
by the controller 16. This flow chart explains operation of 1 embodiment. One [ the 
ignition switch 13 of vehicles ] is checked, and in Step 1, if one [ the ignition switch 
13 ], it progresses to Step 2, and if turned off, this image processing will be ended. 
[0015]In Step 2, it checks whether the shift lever is set as the retreat (R) position by 
the shift sensor 12, when set as the retreated location, it progresses to Step 3, 
otherwise, it returns to Step 1. When the shift lever is set to the retreated location, a 
ranging position is moved to the left end of the center of the sliding direction of the 
imaging range of the camera 1 1 at Step 3. 

[0016]Since the camera 1 1 was installed in the bumper projected to the method of the 
very back of the vehicle rear in this embodiment so that the optic axis 21 of the taking 
lens 11a might become a road surface and parallel as mentioned above, The image of 
the vehicles back picturized with the camera 1 1 turns into an image seen from the 
height of the bumper, as shown in drawing 5 , and the center of an image serves as a 
position along which the optic axis 21 of the camera 1 1 passes. Therefore, the object 
projected on Chuo Line of the sliding direction of an image is an object in the height of 
the bumper of a self-vehicle, and when it retreats, it is the object which contacts first 
the bumper projected at the self-vehicle rear, i.e., an obstacle. 

[0017]So, in this embodiment, along Chuo Line of the sliding direction of the imaging 
range of the camera 1 1 , a ranging position is moved with a prescribed interval from the 
left end of an imaging range to a right end, and the distance to an obstacle is 
measured for every ranging position. 

[0018]Next, the focus of the taking lens 11a is performed so that the focus of the 
image in a ranging position may suit at Step 4. At continuing Step 5, the distance to 
the obstacle in a ranging position is measured, and it memorizes in the memory 15b. 
[0019]Here, based on a video signal, the focus state of the taking lens 1 1a is detected, 
and how to carry out the focus of the taking lens 11a to a ranging position, and 
measure object distance is explained. 

r0020] Drawing 7 shows the physical relationship (a) of the taking lens 11a of the 
camera 11, and an unillustrated CCD sensor (image sensor), and the relation (b) 
between a CCD sensor position and the high frequency component of an image. 
Generally the high frequency component of the video signal outputted from a CCD 
sensor so that clearly from a figure, In the focus state which carries out image 
formation of the sharp object image on a CCD sensor, the taking lens 11a serves as 
the maximum, and it decreases, so that the image formation position of the taking lens 
1 1a is located before a CCD sensor or in back. 



[0021]Then, the high frequency component contained in a video signal is extracted, 
and if the position of the taking lens 11a is adjusted so that a high frequency 
component may become the maximum, the focus of the taking lens 1 1a can be carried 
out to a distance measurement position. For example, as shown in drawing 8 a, when 
the position of a CCD sensor is before a focus position, Namely, when the taking lens 
1 1a is in the state of connecting an image back sharper than a CCD sensor, The focus 
state where the taking lens 11a connects a sharp image on a CCD sensor can be 
acquired by letting out the taking lens 11a toward the front of the direction 1 1 which 
the high frequency component of a video signal increases as shown in drawing 8 b and 
drawing 8 c, i.e., a camera. Although a graphic display is omitted, When the position of 
a CCD sensor is back rather than a focus position (i.e., when the taking lens 1 1a is in 
the state of connecting a sharp image before a CCD sensor), The focus state where 
the taking lens 11a connects a sharp image on a CCD sensor can be acquired by 
rounding the taking lens 11a toward the back of the direction 11 which the high 
frequency component of a video signal increases, i.e., a camera. 

[0022]The distance to a ranging position when carrying out the focus of the taking 
lens 1 1a to the ranging position of an imaging range is found by an operation based on 
a lens position and a focal distance with the taking lens of fixed focus distance. In the 
camera which used the zoom lens for the taking lens, distance is found based on the 
focal distance found from the position of the zooming lens for changing a focal 
distance, and the position of the focusing glass which performs a focus. 
[0023]The direction which zoomed in with the camera provided with the zoom lens, 
and expanded and picturized the narrow imaging range, The boundary of the color of 
an image can become clear, distribution of the high frequency component contained in 
a video signal can become steeper, and a focus can be carried out to a ranging 
position more correctly than the camera of the taking lens of fixed focus distance, 
therefore the distance to a ranging position can be found more correctly. 
["00241 Drawing 9 is a flow chart which shows the routine which measures the distance 
to a ranging position. This flow chart explains the instrumentation method of the 
distance to a ranging position. It asks for the high frequency component HO of the 
image acquired in arbitrary taking-lens positions in Step 11. At continuing Step 12, 
only default value moves a taking-lens position back. It asks for the high frequency 
component H1 of the image after changing a taking-lens position in Step 13. 
[0025]In Step 14, it is checked whether the difference of the high frequency 
components HO and H1 of the image after changing before changing a taking-lens 
position is smaller than the predetermined value epsilon. When the variation of the 
high frequency component of the image before and behind taking-lens movement is 
smaller than the predetermined value epsilon, it progresses to Step 15, It judges that 
the taking lens 1 1a is carrying out the focus to the ranging position, and the distance 
to a ranging position is calculated based on the position and focal distance of the 



taking lens 1 1a in the focus state of connecting a sharp image to a CCD sensor place. 
[0026]On the other hand, when the variation of the high frequency component of the 
image before and behind taking-lens movement is beyond the predetermined value 
epsilon, it progresses to Step 16, and the direction of lens movement is determined 
with reference to the lens movement table shown in drawing 10 , and the taking lens 
11a is moved. At continuing Step 17, the high frequency component HO before lens 
movement is updated by the high frequency component H1 after lens movement, it 
returns to Step 13, and the above-mentioned processing is repeated. 
[0027]After finding the distance to a ranging position, it checks whether the ranging 
processing mentioned above to the right end of an imaging range in Step 6 of drawing 
6Jias been ended, and when ranging processing is not ended to a right end, it 
progresses to Step 9, and only a predetermined interval carries out horizontal 
migration of the ranging position to the right-hand side of an imaging range. Then, it 
returns to Step 4 and ranges to a new ranging position. 

[0028]Although this embodiment shows the example which ranges by moving a ranging 
position to a right end from the left end of an imaging range, it may be made to move 
to a left end from a right end. 

[0029] Drawing 1 1 shows the ranging result of the vehicles back in a motor pool. Every 
prescribed interval is ranged from the left end 25 of the horizontal imaging range 23 to 
the right end 26, and if the distance to the obstacle measured in each ranging position 
is connected in order, the line f 1 as shown in drawing 12 can be drawn. This line f1 is a 
line showing the distance to the obstacle which exists in the height of the bumper of 
vehicles back, and, below, it will be called an "obstacle line." 

[0030]When it ranges from the left end of the imaging range 23 to a right end, the 
distance of each ranging position is read from the memory 10b at Step 7 of drawing 6, 
and it changes in perspective according to the image of the camera 11. 
[0031]Here, drawing 13 - drawing 15 explain how to change the obstacle line f1 
showing the distance to the obstacle of vehicles back into the line seen from the 
viewpoint of the camera 11 in perspective. 

[0032]In drawing 13 , distance from the position f2 of the camera 1 1 to the point I on 
the obstacle line f1 is set to L Based on the camera position f2, the point I on the 
obstacle line f1 is changed into the coordinates of the point P on the image f3 of the 
camera 11. Although the image f3 carries out image formation behind the taking lens 
1 1a of the camera 1 1 actually, the focal position of the taking lens 1 1a is placed on the 
x-y flat surface of z= 0 for explanation, and the image f3 is placed between the 
obstacle line f1 and the camera position f2 here. 

[0033]As shown in drawing 14 , the position f2 of the camera 1 1 is set to (Xc, Yc, Xc), 
and the field angle of thetaw and a perpendicular direction is set to thetah for the 
horizontal field angle of the taking lens 11a. Width of the image f3 picturized with the 
camera 1 1 is set to W, and height is set to H. It asks for the width w on the image f3, 



and height h as a thing corresponding to the point P which is in the position of an 
upper bed to the distance w and the distance h from the left end on the image f3 of 
the camera 1 1 as the point I on the obstacle line f1 which is in the distance L from the 
camera 1 1 shows drawing 15 . 

[0034]First, it asks for the z-coordinate Zi of the point I. A following formula is 
materialized by a Pythagorean proposition. 
[Equation 1] 

Zi is calculated with the (Zi-Zc)*(Zi-Zc)+Yc*Yc=L*L expression 1. Next, it asks for 
the y-coordinate of the point P. A following formula is materialized from physical 
relationship. 
[Equation 2] 

Zs=Yc*tan(pi/2-thetah/2)+Zc, Yps=Yc*(Zs-Zp)/(Zs-Zc), Ype=(Zp-Zc) 

*tan(thetah/2)+Yc, and Ypi=Yc*(Zi-Zc)/(Zp-Zc) 

From this, the distance h from the upper bed of the image f3 to the point P is found. 

namely[Equation 3] 

h=H*(Ype-Ypi)/(Ype-Yps) 

[0035]Similarly, a following formula is materialized also about the distance w from the 
left end of the image f3 to the point P. 

[Equation 4](Xi-Xc)*(Xi-Xc)+(Zi-Zc)*(Zi-Zc)=L*L, Xps=Xc-Zc*tan (thetaw/2), 
Xpe=Xc+Zc*tan (thetaw/2), Xpi=(Xi-Xc)*(Zi-Zp)/(Zi-Zc) 
The distance w is found from this. 
[Equation 5] 

w=W*(Xpi-Xps)/(Xpe-Xps) 

The obstacle line f1 seen from the vehicles upper part is convertible for obstacle line 
f1 ' on the camera image f3 by performing the operation mentioned above about all the 
points I on the obstacle line f1 , and searching for the coordinates (w, h) of the point P 
on the corresponding image f3. 

r0036l Drawing 16 shows obstacle line f1 ' changed in order to put on the image f3 of 
the camera 11. According to the converting method mentioned above, the front 
camera 1 1 side expands a transverse direction of the imaging range 23 of the camera 
1 1 shown in drawing 12 , and it is conversely reduced as it keeps away from the 
camera 11. Simultaneously, a lengthwise direction is reduced as it keeps away from 
the front camera 1 1 side far away, an imaging range after conversion turns into the 
range of 27, and an obstacle line after conversion becomes f1'. In drawing 16, the line 
segment 28 shows a result of having changed a line segment of the same length put on 
a lengthwise direction at equal intervals, and, thereby, can understand a conversion 
result of distance of a lengthwise direction of an imaging range. 

[0037]Next, in Step 8 of drawing 6 , obstacle line f1' after changing into an image of 
vehicles back shown in drawing 5 picturized with the camera 1 1 is compounded, and it 
displays on the monitor 14. At this time, obstacle line fT is displayed in red. An 



example of indication results is shown in drawing 17 . Since the camera 1 1 is attached 
so that an optic axis may become a road surface and parallel against a bumper of a 
vehicle rear in this embodiment, As mentioned above, an object on Chuo Line of a 
sliding direction of an image picturized with the camera 11 is an obstacle of height of a 
bumper, therefore obstacle line f1' becomes a straight line of a center of a sliding 
direction of an image. If the above image processing is ended, it will return to Step 1 
again and processing mentioned above will be repeated. 

[0038]Since a bumper which the self-vehicle rear projected most compounds 
obstacle line fT showing an obstacle which contacts first on an image of vehicles back 
picturized with the camera 1 1 and it was made according to this embodiment to 
display on it as shown in drawing 17 , While distance from a lower end of a back image 
to obstacle line fT expresses distance from a rear bumper to an obstacle, tends to 
hold a sense of distance to an obstacle and retreats, operation in a case of advancing 
to a motor pool, etc. becomes easy. 

[0039]If the red R and middle distance are classified by color in the yellow Y, a long 
distance is classified by color in the blue B and a short distance is displayed, 
corresponding obstacle line f1' to distance to an obstacle as shown in drawing 18 , it 
will become easy to recognize an obstacle near vehicles visually. It may be made to 
change thickness of obstacle line fT according to distance to an obstacle, as shown in 
drawing 19 . It may be made to change a color and thickness of obstacle line fT 
according to distance to an obstacle. Or it may be made to change brightness of a 
foreground color of obstacle line fT according to distance to an obstacle. 
[0040]«A 2nd embodiment of an invention» A 2nd embodiment as which a 
self-vehicle rear periphery detects and displays a position of an obstacle which 
approaches most, and distance to the obstacle is described. Composition of this 2nd 
embodiment is the same as composition of a 1st embodiment shown in drawing 2 , and 
omits explanation. 

[0041] Drawing 20 is a flow chart which shows an image processing program of a 2nd 
embodiment. The same step number is attached to a step which performs the same 
processing as a program shown in drawing 6 , and it explains focusing on a point of 
difference. If ranging ends a Chuo Line top of a sliding direction of an imaging range 
from a left end to a right end at Step 6, it will progress to Step 6A, and a 
closest-approach point of a self-vehicle rear periphery and an obstacle of vehicles 
back is detected. 

[0042]Suppose that the obstacle line f1 as shown in drawing 21 was obtained in range 
measurement to an obstacle of vehicles back mentioned above. f4 in a figure is a 
periphery line of the self-vehicle rear. Information on a fixing position of the periphery 
line f4 of these vehicles and a camera is beforehand memorized in the memory 15b of 
the controller 15. As shown in drawing 22 , while setting up the m points E1-Em on the 
obstacle line f1, the n points C1-Cn are set up on the vehicles periphery line f4. And 



the distance Lij between the arbitrary points Ej Q=1-m) on the obstacle line f1 and the 
arbitrary points Ci 0=1 -n) on the vehicles periphery line f4 is calculated. If the 
distance Lij is calculated to all the combination of the point Ej and the point Ci, the 
distance Lij for two shortest points in them will be searched, and those two points will 
be made into a closest-approach point. 

r0043l Drawing 23 is a flow chart which shows closest-approach check appearance 
processing. In Step 21, the n points Ci 0=1 -n) are set as the m points Ej 0=1 -m) and 
vehicles periphery lines f4 on the obstacle line f1 , respectively, i and j are reset to the 
initial value 1 at continuing Step 22. The distance Lij between arbitrary two-point 
Ci-Ej between the obstacle line f1 and the vehicles periphery line f4 is calculated at 
Step 23, and it stores in distance table T as shown in drawing 24. 
[0044]When checking whether a distance operation has been ended and not having 
ended to the termination Em of j=m f1, i.e., an obstacle line, at Step 24, it progresses 
to Step 26. At Step 26, j is **************ed, it returns to Step 23, and the 
above-mentioned distance data processing is repeated. When the termination Em of 
the obstacle line f1 is reached, it progresses to Step 25, it checks whether a distance 
operation has been ended to the termination Cn of i=n f4. i.e., a vehicles periphery line, 
and when not having ended, it progresses to Step 28. In Step 28, while setting 1 as j, i 
is **************ed, it returns to Step 23, and the above-mentioned distance data 
processing is repeated. 

[0045]When a distance operation is ended in all the combination of the point Ej on the 
obstacle line f1, and the point Ci on the vehicles periphery line f4, the minimum is 
looked for out of a distance operation value stored in distance table T at Step 27. It 
changes into a point on obstacle line f1' which read, the point Ej Ej, i.e., a 
closest-approach point, on the obstacle line f1 corresponding to the shortest distance, 
and looked at it from a viewpoint of the camera 1 1 at continuing Step 30 at Step 29. 
[0046]It changes into obstacle line f1' which looked at the obstacle line f1 from a 
viewpoint of the camera 1 1 with a converting method mentioned above in Step 7 of 
drawing 20 . And as shown in drawing 25 , changed obstacle line f1' is compounded on 
an image of vehicles back picturized with the camera 11, and it expresses on the 
monitor 14 as Step 8. The closest-approach point mark 40 is displayed on the upper 
obstacle line fV closest-approach point Ej, and the window 41 with which distance to 
a closest-approach point of "2.7 m of closest-approach points" is expressed near it is 
highlighted. 

[0047]According to this embodiment, a position of a high obstacle of a possibility that 
a self-vehicle rear periphery will contact, and distance to that obstacle can be 
recognized at a glance, it is easy to hold a sense of distance to an obstacle, and 
contact with an obstacle can be avoided certainly. 

[0048]«A 3rd embodiment of an invention» A moving track of vehicles is predicted 
and a 3rd embodiment that detects and displays a position on an obstacle line which a 



vehicles periphery line after prescribed distance movement approaches most is 
described. 

r0049l Drawing 26 shows composition of a 3rd embodiment. The same numerals are 
attached to the same apparatus as composition of a 1 st embodiment shown in drawing 
2, and it explains focusing on a point of difference. The rudder sensor 50 detects the 
steering angle theta of a steering wheel. 

[0050]The controller 15 predicts a position of vehicles periphery line f4' when 
prescribed distance retreat is carried out with the present steering angle theta from a 
position of the present vehicles periphery line f4, as shown in drawing 27 . And vehicles 
periphery line f4' in a predicted position and the obstacle line f1 detect and display a 
point (closest-approach point) of approaching most. An arithmetic method of this 
closest-approach point omits explanation from ** which is the same method as a 2nd 
embodiment mentioned above. 

r0051l Drawing 28 is a flow chart which shows an image processing program of a 3rd 
embodiment. The same numerals are attached to a step which performs the same 
processing as a program shown in drawing 6 , and it explains focusing on a point of 
difference. If ranging from a left end to a right end ends a Chuo Line top of a sliding 
direction of an imaging range at Step 6, it will progress to Step 6B. and the rudder 
sensor 50 detects the steering angle theta, and a position of vehicles periphery line f4' 
when prescribed distance retreat is carried out from a position of the present vehicles 
periphery line f4 with the present steering angle theta is predicted, next — a step — 
six — a — a predicted position — it is — vehicles — a periphery — a line — f — 
four — ' — an obstacle — a line — f — one — most — approaching — a point 
(closest-approach point) — detecting — an obstacle — a line — f — one — a top — 
a closest approach — a point — a camera — 1 1 — a viewpoint — from — having 
seen — an obstacle — a line — f — one — ' — a top — a point — changing . 
[0052]It changes into obstacle line fV which looked at the obstacle line f1 from a 
viewpoint of the camera 1 1 with a converting method mentioned above in Step 7 of 
drawing 28 . And as shown in drawing 29 , changed obstacle line fV is compounded on 
an image of vehicles back picturized with the camera 11, and it expresses on the 
monitor 14 as Step 8. The closest-approach point mark 40 is displayed, respectively 
on the closest-approach point Ci on obstacle line f1 'the upper closest-approach 
point Ej and the prediction vehicles periphery line f4\ and the window 41 1 with which 
distance between the closest-approach points Ej and Ci of "1.6 m of 
closest-approach points" is expressed near it is highlighted. 

[0053]According to this embodiment, after carrying out prescribed distance 
movement with the present steering angle, a position of a high obstacle of a possibility 
that a self-vehicle rear periphery will contact, and distance to that obstacle can be 
recognized at a glance, it is easy to hold a sense of distance to an obstacle, and 
contact with an obstacle can be avoided certainly. 



[0054]Although each embodiment mentioned above shows an example which installs 
the camera 11 in a vehicle rear, and picturizes and displays an image of vehicles back 
required at the time of parking. A camera is installed in the front, the side, or rear side 
of vehicles, also when picturizing and displaying an image of the front of vehicles, the 
side, and slanting back, this invention can be applied, and same effect is acquired. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

fDrawing 1] It is a figure explaining the problem of a Prior art. 

fDrawing 2] It is a figure showing the composition of a 1 st embodiment of an invention. 
fDrawing 3] It is a figure showing the imaging range of the camera seen from the 
vehicles side in a motor pool. 

fDrawing 4] It is a figure showing the imaging range of the camera seen from the 
vehicles upper part in a motor pool. 

fDrawing 5] It is a figure showing the image of the vehicles back picturized with the 
camera in the motor pool shown in drawing 3 and drawing 4. 

fDrawing 6] It is a flow chart which shows the image processing program of a 1st 
embodiment. 

fDrawing 7] It is a figure for explaining the focusing method of a camera. 

fDrawing 8] It is a figure for explaining the focusing method of a camera. 

fDrawing 9] It is a flow chart which shows the routine which measures the distance to 

a ranging point. 

fDrawing 10] It is a figure showing a lens movement direction table. 

fDrawing 11] It is a figure showing the ranging result of the vehicles back in a motor 

pool. 

fDrawing 12] It is a figure showing the obstacle line seen from the vehicles upper part. 
fDrawing 13] It is a figure for explaining how to change the obstacle line f1 seen from 
the vehicles upper part into obstacle line f1' seen from the viewpoint of a camera. 
fDrawing 14] It is a figure for explaining how to change the obstacle line f1 seen from 
the vehicles upper part into obstacle line fT seen from the viewpoint of a camera. 
fDrawing 15] It is a figure for explaining how to change the obstacle line f1 seen from 
the vehicles upper part into obstacle line fT seen from the viewpoint of a camera. 
fDrawing 16] It is a figure showing obstacle line fT changed in order to put on the 
image of a camera. 

fDrawing 17] It is a figure showing an example which compounded and displayed the 
obstacle line on the image picturized with the camera. 

fDrawing 18] It is a figure showing an example which classified by color and displayed 



the obstacle line according to the distance to an obstacle. 

[Drawing 19] It is a figure showing an example which changed the thickness of the 
obstacle line according to the distance to an obstacle. 

[Drawing 20] It is a flow chart which shows the image processing program of a 2nd 
embodiment. 

[Drawing 21] It is a figure showing the relation between an obstacle line and a vehicles 
periphery line. 

[Drawing 22] It is a figure for explaining the detecting method of the closest-approach 

point of a self-vehicle rear periphery and the obstacle of vehicles back. 

[Drawing 23] It is a flow chart which shows closest-approach check appearance 

processing. 

fDrawing 24] It is a figure showing a distance table. 

fDrawing 25] It is a figure showing an example which carried out the composite display 

of an obstacle line and the closest-approach point to the image of vehicles back. 

fDrawing 26] It is a figure showing the composition of a 3rd embodiment. 

fDrawing 27] It is a figure for explaining how to predict the position of a vehicles 

periphery line when prescribed distance retreat is carried out. 

fDrawing 28] It is a flow chart which shows image processing of a 3rd embodiment. 

fDrawing 29] It is a figure showing an example which carried out the composite display 

of the closest-approach point of an obstacle line and a prediction vehicles periphery 

line to the image of vehicles back. 

[Description of Notations] 

11 Camera 
11a Taking lens 

12 Shift sensor 

13 Ignition switch 

14 Monitor 

15 Controller 
15a CPU 
15b Memory 

21 Optic axis 

22 A vertical imaging range 

23 A horizontal imaging range 

24 Wall 

25 The left end of a horizontal imaging range 

26 The right end of a horizontal imaging range 

27 The imaging range after conversion 

28 Line segment 

40 Closest-approach point mark 

41 Window 



50 Rudder sensor 
f1 Obstacle line 

The obstacle line seen from the viewpoint of an fY camera 

f2 Position of a camera 

f3 Image picturized with the camera 

f4 Vehicles periphery line 

f4' prediction vehicles periphery line 
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^T£?tc, »«»*T086«K:jEi;TP»*ft^-<V 
f 1* <fc 3 teLTfe«fcl/\, 3 6*^ 

ic/scrBS^-of r ^WM^^^J: 
[0040] mm(om2CDmffi(DBm» as«atfw» 
«ajbTa^T**2oiifli<oflg«*KW , Tao 

clOS2<Ollfll<0»lBO««ttBI2JC^T» 1 

[0041] 020 a, S2 (ommmm^mm^mv 

* £ Tffl'JSitf *l 7 Lfc£>X7^y:/6A 'Nit*, g 
[0 0 4 2] ±2bft*W«*Ol»««*'t*OffiI«Wffi 

tc43v>r, 02 iu:as-T*5fti»»*5-r>f 

50 ffl&^a&rn^bn— 7— 1 1 5btcfB'lf>LT 

*5<o 0 2 2^T£*5^ Bfft^Vf li^mfli 
£>£E l-EmSRStStktlc, *W*WS*§tf 4_b 

yf i±Ott*OjSE j (j = l-m) £*W*WS»f 

4±osmc i (i-i-n) tcomozmh i j 

^ift§ 0 jSE J £/&C i oT^TOffl^-&t)^tc» 
LT8B«L i jfciRULfcSK *n&0*T«fcJS^2 

»iiL i j *«*su *ne>o2js*«8Bfi/£4: 
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40 [0 0 4 3] 0 2 3tt«»3e^*Uiffiffl**T'7n— ^« 

l±fcmffl<Dj£E J (j = l~m) i:*SWW8«f4K: 
nfflOjSC i (i = l~n) ^JJVfftK^TSo $K 
Xf77 P 2 2tM fcj**B»illC'J-b7htSo Xf 
7^2 3tBffl7^fyf 1 fc*W^JB»f 4 ^OP^<0 
«3ft^2^C i ~E j MOEML i j ^Sl, 02 4 
tc^Tcfc -5 ^SBWt-r-^l/T tcteJflTSo 
[0 0 4 4] XT772 4t j = m2)\ o£ DPt^ft^ 
-Of 1 0««EmStKlli«WS:»7bftA^5^ 
50 BSBU $?7lT^^t[iXf77 0 2 6^\itt?o X 
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*87I/fefcitts Xf7^2 7;l/Tfc:ffi 

72 9^«/hffi«fcWjS'r*i»*»9-ry f i±^e 

[0 0 4 6] 02 0<D7sTVZf7Kl5^T, ±iSb;fe£ 
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<D%%Tfftfemmmvrct%(DmnHmMf 4' ofe 
■wsitSa fit, ^»jffi«jcifcs*w^jai»f 

4 f fcBMMfc^^f 1 fc#«fc«8fi , r*jft 
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^-1 4tc«^-r^o set:. »«*7^yf r ±0 
•sbsj^e j t^mmm^mmf 4' ±©*8sfi£c 1 

±fc:*ti J Pn«»5S/iSv— > 4 0*a^U *05S<k: 
r*SS&£l. 6mj i:V^S!$ME j fcC i OBO 

20 [0 0 5 3] CO*SS<DflgJ»tcJ:tltf, SffiOlMeftO 
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[0 1 9] »*1fc*^Offi«|k:jSi:TI»»W9-YV<0 
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